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Abstract 

Polyhedral geometry can be used to quantitatively assess the dependence of rank- 
ings on personal preference, and provides a tool for both students and universities to 
assess US News and World Report rankings. 

1 Introduction 

The annual US News and World Report (USNWR) rankings have become a 
major factor influencing college admissions [1]. University placements affect the 
decisions of students, and the rankings are carefully monitored by the institu- 
tions that are compared. Administrators tout the rankings when they are useful 
for recruitment and fundraising ^ while at the same time lamenting perceived 
flaws in the methods used to gather data and establish rankings. They also be- 
moan the fluctuating pressures exerted on their institutions as rankings change 
from year to year [3] . 

The current USNWR rankings of top national universities are based on ag- 
gregate scores computed for 130 universities. The scores consist of linear combi- 
nations of measurements of several attributes, such as prestige, selectivity, stu- 
dent graduation and retention, faculty resources, and student satisfaction [4]. 
Criticisms of USNWR have addressed many of the ingredients that contribute 
to the final rankings. For example, there has been widespread disagreement 
on what type of data should be collected. Furthermore, even when there has 
been some agreement that certain attributes of universities are informative for 
comparison, there has been disagreement on how to perform the measurements. 
Finally, there is still no consensus on how to combine the various scores to obtain 
a single number reflecting the overall appeal of a university. 

This last point is the focus of our study. We maintain that any information 
collected about a university is valuable, although its relative importance to 
individuals may differ based on their personal preferences. Certain students may 
value the prestige afforded to selective schools, while others may be concerned 



with student/faculty ratios or class size. Similarly, universities may wish to 
advertise their strengths to targeted groups who may best appreciate them. The 
need to aggregate scores is based on the difficulty in comparing multiple features 
for many schools. However instead of a single weighting, we believe it is useful 
and informative to examine multiple options. Indeed, USNWR has admitted 
as much. In a National Opinion Research Center study funded by USNWR to 
evaluate its methodology [5] it was reported that The principal weakness of the 
current approach is that the weights used to combine the various measures into 
an overall rating lack any defensible empirical or theoretical basis. 

According to USNWR methodology [1] , seven main categories of attributes 
comprise the final aggregate score: 25% peer assessment, 20% freshman reten- 
tion, 20% faculty resources, 15% student selectivity, 10% financial resources, 5% 
graduation performance, and 5% alumni giving. Various measurements for each 
attribute form the basis for assigning each national university a numerical score 
in each of the seven categories. The seven numerical scores are then weighted 
and summed with the above listed weights to give the universitys final score. 
We studied how universities final scores and rankings depend on the relative 
weights assigned to the seven categories. We restricted our attention to nonneg- 
ative weights only, since each of the seven categories is measuring a desirable 
feature. We note that our methods are not restricted to just seven categories, 
although computational limits may prohibit analyses for too many attributes. 
In such cases, the methodology of [5] can be used to first identify correlates 
among attributes and thereby reduce the dimension of the problem. 

Our study required the seven category scores for each university. USNWR 
publishes three of the category scores, and reports only the rankings for the 
other four categories [7]. We therefore reverse-engineered the four unknown 
categories, from the final scores and category rankings published by USNWR. 
This was accomplished by casting the problem in terms of feasibility for a linear 
program. We also simulated random data for the seven categories and repeated 
our analysis to verify that our findings are typical for rankings of multidimen- 
sional data. 

The mathematical problem of determining the sensitivity of rankings to 
changes in the weights is the geometric problem of determining the vertices and 
normal fans of certain polytopes called /c-ordertopes. A seven-dimensional vec- 
tor describes the seven attribute scores assigned to each university. The convex 
hull of all universities 7d vectors is a polytope, whose vertices are all the possible 
top-ranked universities that can arise for some choice of weights. Similarly, the 
fc-ordertope is a polytope that can be constructed whose vertices represent the 
top k universities that can arise for some choice of weights. The normal fan of 
a fc-ordertope gives the complete partition of scoring weights according to the 
top k ranking they induce. 

We computed the non-negative envelopes of /c-ordertopes for all 7 categories 
in USNWR for k < r. For k = 4 this computation involved finding the non- 
negative envelope of more than 270 million points in seven-dimensional space. 
This was accomplished using Theorem 2.7 and Lemma 4.1, in addition to novel 
software. We found that there are a total of 27 universities that can appear 
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among the top four for some choice of nonnegative weights. However, only nine 
appear in the top four for more than 5% of the possible weights. These are 
Princeton, Harvard, Yale, Stanford, University of Pennsylvania, Caltech, MIT, 
Duke, and Dartmouth. Any generic ranking is therefore likely to identify these 
top schools. In [2 it was reported that rankings influence college choices 
especially for top-ranked universities. Thus, it is noteworthy that the order of 
the top-ranked universities can vary for a significant fraction of weights. For 
example, we found that Yale (USNWR #3) beats Princeton (USNWR #1) for 
26% of weight choices. Although exact order is sensitive to weight choices, both 
Yale and Princeton (and Harvard) are ranked in the top 5 for 95 percent of 
weight choices; the USNWR data confirms they are top schools irrespective 
of the weights chosen. However for many universities, no single ranking is 
informative. We computed each universitys ranking interval (RI), which is the 
smallest range of ranks for the university which are induced by at least 95% 
of the possible choices of scoring weights. The average interval width of a 
universitys RI was 35.5; for universities in USNWRs top 25 the average RI width 
was 11.1, and the average RI width for the remaining 105 universities was 41.3. 
Some schools, such as Georgia Tech (RI of [25, 73]), have extremely large RIs 
compared to similarly ranked schools, meaning that their specific placement by 
USNWR is essentially arbitrary. 

The ranking interval measures the smallest range of ranks induced by at 
least 95% of possible scoring weights. As the average RI interval widths show, 
significant instability in ranks is the rule rather than the exception. We found 
that all but the top three universities ranked by USNWR (Princeton, Harvard, 
Yale) had an RI of six or more ranks. This confirms that while the top universi- 
ties are good all around regardless of the scoring weights, the rankings of other 
universities depend strongly on individual preferences. 

The proliferation of college rankings [9] has created a confusing situation 
where multiple rankings that are difhcult to compare and contrast are circulated, 
frequently without the underlying raw data, fc-ordertopes organize attribute 
score vectors in a way that reveals the dependence of rankings on weights. 
They allow for analysis of the robustness of individual placements, and for 
aggregate statistics to be compiled. Furthermore, the examination of an entire 
fc-ordertope provides a novel way to compare and contrast universities. The 
volumes of the normal fans of fc-ordertopes also provide a quantitative basis for 
selecting weights that match desired rankings. 

We believe that the data collected by USNWR is useful for both students 
and universities, but ideally USNWR will publish the raw measurements and 
leave the rankings to the reader, as is done for German universities by the 
CHE/DA AD consortium [TU]. The CHE/DAAD consortium provides a website 
where users can explore the strengths and weaknesses of universities by specify- 
ing preferences in weights before rankings are displayed online. Furthermore, in 
the USNWR magazine a few different rankings could be published for a handful 
of popular preferences. The publication of a single ranking provides readers with 
a summary that may be misleading and fails to properly account for personal 
preference and uncertainty in weights. By examining fc-ordertopes, students can 
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also account for the uncertainty in their preferences in a systematic way, and 
universities can identify the spectrum of weights where they rank highly so that 
they can better target prospective students. 

2 Polyhedral geometry 

In this section we introduce the basic definitions and theorems from polyhedral 
geometry necessary for our results. We refer the reader to [13j for more details. 

Definition 2.1. The affine hull of a set of points V = {wi, . . . , w„} C R'' is 

the set {X^Li Ci^^'i I I^Li = 

Definition 2.2. The convex hull of a set of points V — {vi, . . . ,Vn} C R'' is 
the set {J2i=i Ci"^* I 127=1 = 1> Vc* > 0}. 

Definition 2.3. A polytope is a convex hull any finite non-empty V C R"*. 

Equivalently, a polytope can be defined as a bounded polyhedron, where a 
polyhedron is the set of solutions of finitely many weak linear inequalities. 

The dimension of a polytope P C R*^ is the dimension of its afhne hull as 
a manifold. To avoid confusion between d and dim P, d is called the ambient 
dimension of P. 

Definition 2.4. Given a polytope P C R'' and a functional c E R"^, the face 
Fc <Z P is the set argmax^gpc • x. 

Any face of a polytope is again a polytope. Note that P is a face of itself 
by putting c — 0. For generic choices of c, the face Fc will be a single point, 
which is called a vertex of P. 1-dimcnsional faces are called edges, (dimP — 1)- 
dimensional faces are called facets, and (dimP — 2)-dimcnsional faces are called 
ridges. 

Definition 2.5. Given a polytope P C R'' and a face F C P, the normal cone 

of F is the set of all functionals c for which Fc = F. 

The relative interiors of normal cones of all faces of P partition R*^, and 
form a lattice ordered by inclusion. The set of normal cones has other special 
properties as well, and is called the normal fan of P. 

Definition 2.6. Given a polytope P C R'' and a matrix A £ R™^*^, let C = 
{c I Ac>0}. The A-envelope of P is the set of faces {F C P\ iV(P)nC ^ 0}. 
If A~ I, then the A-envelope is called the non-negative envelope of P. 

The main objects we compute and analyze are polytopes called k-ordertopes. 
For each fc, the vertices of the fc-ordertope correspond to top fc university rank- 
ings which are attainable for some choice of weights. The fc-ordertope is de- 
fined as follows: Consider a set of vectors Ui that are distinct (we work with 
the score vectors from the US News data). Fix positive decreasing weights 
Oil > a2 ■ ■ ■ > Oin > where n is the number of universities. The k-ordertope 
Pk is the convex hull of all weighted sums of k universities' score vectors: 

k 

Pk = conv{^Q!iUo-Jo-i,...,o-fc 

1=1 
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where cti, (T2, • • • Cfe range over all possible n{n — 1) ■ (n — k + I) choices of k 
distinct indices. Each vertex of the A:-ordertope corresponds to a choice of top 
k universities which is obtainable for at least one choice of scoring scheme c: 

Theorem 2.7. A point Yl'i=i '^i'^at € Pk is a vertex of Pk if and only if there 

is linear scoring scheme c which satisfies c ■ > c • ti^^ > ... > c • Uc^, and 
c ■ Uc^. > €■ Uj for all j ^ {ai, . . . , cr^}. The normal fan of Pk is independent of 
the choice of weights at . 

Proof. Suppose c satisfies c • u^i > ... > c • , and c • u^^ > c - uj for all 
j ^ {(Ti, . . . , (T/j}. Then we claim that c • X^^L^ aiU„^ > c ■ Y^^^i ctiU-y^, unless 
7i = (7i for all i. By basic convexity, this would imply that X^j^^ ^iWaj is a 
vertex on Pk- 

So consider any choice of distinct 71, 72, • • • 7fe- 

• If c • u.y^ < c • for some i < j, then since > aj we have 

c • {aiUj. + ajU-y.) > c ■ {aiU^- + aju^.) 

So by swapping the values of 7j and 7^, we can obtain a new point in Pk 
which has a higher dot-product c • ^ 

• If c • u.yi > c • > ... > c • u-),^ , and if 7fc ^ {ai, . . . CTfc}, then there 

must also be some ai ^ {71, . . . , 7^} by basic counting principles. We have 
c • w<Ti > c • McTj, > c ■ M-^j. , and hence, by redefining 7^ := ai, we can obtain 
a new point in Pfe which increases the dot-product c • J2i=i oa'^-yi- 

Thus, X]i=i Q^i'^Ti cannot maximize the functional c over Pfc, unless c-u^^ > 
c • u-y2 > ... > c • M.y^ and jk G {""j}- But then all 7^ must be contained 
in {ai}, and so {7^} = {ai}. Furthermore, since c • u-y^ > ... > c • w-y^. and 
c • Ucri > . . . > c • UcTfc , we must also have ji = ai for all i. 

This completes the proof of the 'if part of the theorem. The 'only if follows 
from the 'if: If X/i=i c^i^o-i G Pfe is a vertex, then it must strictly maximize 
some functional c, and this c must induce an ordering c • u^-^ > . ■ . > c ■ Uj^ 
on the score vectors, where the first k inequalities are strict. Then X]i=i cki'^t; 
is the vertex which maximizes the functional c over Pk, via the 'if part of the 
theorem. The proof of the 'if part also shows that X^iLi Q^i"<Ti cannot equal 
this vertex unless ai = % for all i = 1, . . . , fc, as desired. 

In fact we have proved that c is in the relative interior of a vertex normal cone 
^iYli=i otiUai) iff c-Ucti > . . . > c-Mo-fc and c-u^^ > c-uj for all j ^ {ai, . . .,ak}. 
As a corollary, we have that the interiors of vertices' normal cones depend only 
on the Ui and are independent of the choice of a,. This implies that the entire 
normal fan is independent of the choice of Ui as well. □ 

Our theorem characterizes normal cones of Pk and shows why they are im- 
portant. For each choice of k distinct universities u^-^ , ■ ■ ■ , i the normal cone 
^(^i=i oti'^ci) gives all possible functionals which make the ith ranked uni- 
versity for i = 1, 2, . . . /c. Thus Pk and its normal fan completely describe how 
the top k rankings depend on the choice of functional c. 
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Wc arc only interested in reasonable vertices, i.e. vertices whose normal 
cones' interiors intersect the non- negative first orthant R>o- These vertices 
give precisely the top k rankings which are attainable under a reasonable scoring 
scheme, in which no categories are penalized. Since we are not interested in any 
other vertices, we compute the non-negative envelope of fc-ordertopes, instead 
of the entire fc-ordertopes. 

3 Data 

We studied the US News and World Report's Best Colleges 2008 rankings of 
national universities. This data set comprises numerical scores for each of 130 

universities, in seven categories: 

• Peer assessment (university officials are asked to rate the 130 universities 
on a scale from 1 to 5), 

• Freshman retention rate, 

• Graduation performance (number of percentage points above or below the 
imivcrsity's expected graduation rate as predicted by USNWR), 

• Faculty resources (class size, faculty to student ratio, faculty salaries), 

• Student selectivity (SAT scores, application rejection rate), 

• Financial resources, 

• Alumni giving (percentage of alumni that make donations). 

The data can therefore be described by 7-dimensional vectors of measurements 

{ui, i ~ 1, . • . ; 130), which we call the score vectors. In some cases individual 
category scores are obtained by combining several different measurements. We 
do not analyze the data at that level of detail, and focus on the 7 categories 
that are used to produce the overall scores. 

In order to aggregate a university's score vector into an overall score for the 
university, a linear scoring scheme is used where each university Ut is assigned a 
score of c-Ui, for some fixed choice of c G R^. All categories under consideration 
are assumed to be intrinsically 'good', so c is assumed to have non-negative en- 
tries. We can also assume, without loss of generality, that the entries in c sum to 
1. US News uses the specific choice of c = (0.25, 0.20, 0.05, 0.20, 0.15, 0.10, 0.05). 

USNWR does not make available the score vectors Ui, but it does release 
three of the components (peer assessment, graduation performance, and alumni 
giving), as well as the universities' overall scores c • w,j. USNWR also publishes 
the rankings of the universities in each of the four unpublished categories. 
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4 Methods 



4.1 Algorithm to compute non- negative envelopes of k- 
ordertopes 

Our definition of the fc-ordertope Pk provides an oracle to find a vertex of Pk 
wliicfi maximizes a given functional c. Namely, the oracle sorts all Ui according 
to the value of c • Ui, and then returns oa'^i- (In case of ties c ■ Ui = c ■ Uj, 

the lexicographically smaller of Ui and Uj is ranked higher.) 

Given such an oracle for finding vertices, the software iB4e fTT builds the 
entire polytope by carefully choosing functionals and querying the oracle to find 
vertices, until all vertices are guaranteed to be found. iB4e constructs both the 
vertices and facets of the final polytope. It runs in polynomial time when the 
dimension of the polytope is fixed, and the number of oracle queries is exactly 

V + F, where V is the number of vertices of the constructed polytope and F is 
the number of facets. 

Since we are only interested in reasonable vertices, we modify the oracle so 
that it can also return points of the form —Ncj, where is a standard basis 
vector in R'', and iV is a large positive number. Then the outputted polytope 
is the convex hull of P^ U {—Nei, . . . , —Nej} instead of Pk- UN is sufficiently 
large, then this polytope gives the non- negative envelope of Pk, as the following 
lemma shows. 

Lemma 4.1. Let P C R'' be a d- dimensional polytope, and let {ej} be the 
standard basis. Define Qn — conv{P(J {—Nei, ... — Ned})- If N is sufficiently 
large, and if v ^ P is a vertex such that N{v) H R>o 7^ 0, then v is a vertex of 
Qn- If N is sufficiently large and v € P is a vertex of Qn, then v is a vertex 
on the non-negative envelope of P. 

Proof. Suppose u is a vertex of P for which there is c G N(v) n R>o- L'^t 
e — mini Ci- Then e > and we have c{—Nej) < —Ne for all —Ncj. So if N is 
large enough, we have cv > c{—Nej) for all —Ncj, and thus f is a vertex of not 
only P but also Q^. This proves the first half of the theorem. 

Now suppose ti G P is not a vertex on the non-negative envelope of P. Then 

V cannot weakly maximize any non-zero functional c G R>o''- I3y the Farkas 
Lemma, there must be a point x G P such that Xj > Vj for all j = 1, . . . d. Thus 

V — X + tj{~Nej), where all Cj are positive. We rewrite this as an affine 
combination of x and the —Ncj: 

v = {l-5)x + ^(e, - 5x,/N){-Ne,) (1) 
j 

where 5 = f-j)l{'^ + ^j[xjlN)). If N is large, then the ej are small, and 
5 ~ ej will also be small (and positive since the ej are positive). Furthermore 
if N is large then all coefficients in equation ([T]) for v are positive, so equation 
([1]) expresses w as a strictly convex combination of x and the —Ncj. Thus v is 
not a vertex of Qjy for large N (in fact, v is an interior point). □ 
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4.2 Measuring normal cones 

Absent any canonical choice of scoring scheme c, we measure the fraction of 
non-negative scoring schemes which give each ranking. This means measuring 
normal cones of fc-ordertopes, restricted to the first orthant. Since all functional 
along a common ray give the same ranking, we measure a cone with respect to 
the Gaussian density / restricted to the first orthant: 

/(c) ~ e-ll'=ll' if c> 0, /(c) = otherwise 

Geometrically, this is equivalent to intersecting a cone with the unit hyper- 
sphere and first orthant, and then computing the volume of the intersection 
region, which is a spherical polytope. 

Computing exact volumes of spherical polytopcs is difficult in dimensions 
four or higher. But we can easily sample from the first orthant with respect 
to the Gaussian density. This allows us to approximate cone volumes quite 
well for all normal cones whose volume is not too small. This is sufficient 
for our purposes since we are not interested in accurately approximating the 
probabilities of highly unlikely rankings. We can in fact accurately approximate 
the volume of every normal cone though, by approximating a spherical polytope 
by a simplicial mesh. 

This gives a rational alternative to using just one (subjective) choice of c. 
For example, for two universities ?ii,W2 in the 1-ordcrtopc, if N{ui) Cl R>o^ is 
larger than iV(M2) nR>o'', then there is a larger percentage of choices of c which 
make ui the top-ranked university instead of U2. 

We use volumes of normal cones to compute ranking intervals for each uni- 
versity. The 95% ranking interval for university Ui is the shortest contiguous 
interval of ranks I such that at least 95% of functional assign Uj a ranking in 
I. In case a university has multiple 95% ranking intervals of the same width, 
we choose the leftmost interval containing the smallest ranks. 

4.3 Reverse-engineering of unpublished categories 

For each university, the seven category scores computed by USNWR give a 
vector Ui G R"^. We only know unjUi^ and u^y corresponding to peer assessment, 
graduation performance, and alumni giving. However USNWR does publish 
rankings in the other four categories. For example Harvard (USNWR overall 
rank ^2) is ranked #1 in retention and Princeton (USNWR rank ^1) is ranked 
^2 in retention. Thus, even though we don't know the retention scores W22 for 
Harvard and U12 for Princeton, we do know that Harvard's retention score is 
greater than Princeton's, i.e. tt22 > W12 + e for some fixed e > 0. USNWR also 
publishes the overall final score c • Ui for each university. 

Thus, there is a system of linear equations and inequalities that the unknown 
category scores must satisfy. The set of solutions to these linear constraints is 
a polytope. However, since there are 130 universities and four unknown cate- 
gories, the polytope of possible values for the unknown scores is several-hundred 
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dimensional, so we cannot compute the entire polytope explicitly. However stan- 
dard linear programming can quickly find vertices of the polytope that maximize 
given functionals. 

We could use linear programming to find a vertex of the polytope and use 
the vertex as an estimate of the unknown category scores. However, vertices 
are extremal points of the polytope, whereas ideally we want interior points. 
In theory there are fast approximate sampling algorithms for high-dimensional 
polytopes, but they are complicated and to our knowledge none have been 
implemented. Instead we computed many vertices and averaged them to obtain 
a central point. 

In order to assess the quality of our predicted category scores, we treated one 
of the three known categories (alumni giving) as an unknown and therefore were 
able to use it as a control in order to measure the quality of our estimates. We 
therefore estimated category scores for five unknown categories. We minimized 
and maximized each university's unknown category score separately, which re- 
quired solving 5 X 130 x 2 = 1300 linear programs. We then averaged the 1300 
vertices to obtain our estimate for the five unknown categories. 

We then compared our estimate a S R^'^" for alumni giving to the true 
alumni giving scores a. After standardizing so that a, a have standard deviation 
1, we found that \a — a\ was 0.0991 on average. This is not surprising because a 
and a are in the same order — so they are already highly correlated. To see if 
our estimate a was significantly good, we sampled random values for the entries 
in a from the normal distribution with the same mean and standard deviation 
as a, and then sorted a to be in the same order as a. Out of 10000 trials, 95% 
of the random a produced a value of ■mean\a — a\> 0.168, the average value of 
mean\a — a\ was 0.236, and the lowest value of meanja — a\ was 0.10210. 

5 Computational results 

Using our reverse-engineered estimates of the USNWR data, we computed the 
non-negative envelope of fc-ordertopes for k up to 4. For fc = 4, our computation 
entails finding the non-negative envelope of 130 • 129 • 128 • 127 > 270, 000, 000 
points. Such a large computation would be essentially impossible by conven- 
tional means of listing the points and determining which points lie on the enve- 
lope. Thus our use of vertex-finding oracles and iB4e was cruical to accomplish 
the computation. The number of vertices of the fc-ordertopes which we found 
on the boundary of the non-negative envelope are given in Table [S] Note that 
there are over a thousand possible top 4 rankings that USNWR could have 
conceivably published by using a different choice of non-negative weights. 

The schools which can be ranked #1 are Princeton, Harvard, Yale, Stanford, 
U Penn, Cal-tech, MIT, Notre Dame, Johns Hopkins, and Penn State. All of 
these are USNWR top 20 schools except Penn State (USNWR rank #48). Penn 
State is able to be ranked #1 by virtue of its high graduation performance — 
which is a category weighted very low in USNWR's scoring scheme. 

We further found that 17 schools can appear in the top 2, 26 schools can 
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k # Vertices 
~ 10 

2 59 

3 276 

4 1082 

Table 1: Number of vertices of the fc-ordertopes. 

appear in the top 3, and 27 schools can appear in tlio top 4. 

We show an example of part of the normal fan of a 3-ordertope in Figure 1. 
In the example we have restricted the analysis to three of the features weighted 
highest by USNWR (peer assessment, retention, student selectivity) so that the 
picture is 3- dimensional. Each choice of weights is a vector that lies inside one 
of the cones that form the normal fan of the 3-ordertope. The top 3 universities 
for each choice of weights label the cones. For example, with a high preference 
for selectivity, the ranking will be 1. Harvard 2. Yale 3. Princeton, whereas 
a combined preference for peer assessment and freshman retention results in 
1. Harvard 2. Princeton 3. Stanford. The normalized volumes of the cones 
measure the fraction of weights for which each ranking is optimal. 

We then sampled functional c from the non-negative first orthant using 
the Gaussian density, and computed the rankings that the functionals induced. 
Geometrically, this gives approximate volumes of the normal cones of the k- 
ordertopes. 2, 000, 000 functionals were sampled and we computed various sum- 
maries of the induced rankings. We recorded how many times each university 
was assigned each rank between 1 and 130, and how many times university i 
was ranked higher than university j for all pairs 

For all 130 universities we computed ranking intervals, which are given below: 
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University 



USNWR Ranking Ranking CI 
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1 


\1 41 
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fl 41 


Yale Universitv (CT) 

X cxiLK^ w XXI V cx OX u y \ v_/ _i_ ; 


3 


fl 51 


StflTiforH TTnivprsitv 
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4 


\3 81 


University of Pennsylvania, 


5 


f3 101 


Cailifornia Institute of Technology 
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2 21 


VTasRarhnRPttR Institute of TVrlmoloffv 
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2 15 


l~)nlce TTniversitv (INFr^i 


8 


\2 111 


Oolnvnbia Universitv TNYl 

v_^\_/x i^xxx i^xcxp xxxv^^xoxijy \-^' ^ / 


9 


\6 131 


University of Chicago 


9 


7 18 


Dartmouth College (NH) 


11 


fS 121 
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12 


f5 191 


Cornell Universitv (IVY^l 

v^v^x xxoxx w XXX V ox ox u y \xi x / 


12 


f9 151 


Rrown Universitv (PIi 

j_/x \j vv XX \j XXX V ox ox u y \ -L v-I- / 


14 


9 18 


ISIortlrwestern Universitv (IIil 

X 1 Kji. uxxyy v_,o uox xx xxx v ox ox u v \ J-J_j / 


14 


fn 191 


Johns Uonkins Universitv flVTDl 

^ vyxxxxo X j-v/ L/x\.xxxo \_y xxx V ox ox u y ixvxj—^ i 


14 


f4 181 


Rice University (TX) 


17 


fl3 211 


Pmnrv TTnivor^ifv i(tAi 
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17 


14 35 


Vanderbilt Universitv fTNl 

V ctxxvxox k_/j.x u O' xxxvoxoxuy i j^xi i 


19 


16 26 


University of Notre Dame (IN) 
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5.1 Validation of results on random university data 

Since wc could not perfectly reverse-engineer the university score vectors from 
the USNWR data, we simulated data to verify that our results are typical for 
rankings of seven-dimensional data. To generate the ith university's data u,, 
we first sampled an intrinsic goodness G R from the normal distribution. We 
then set 

Uij =pU + (1 -p)eij 

where each e^j is sampled from the normal distribution and p E [0,1] is a fixed 
constant. Thus the seven category scores in Ui are correlated and influenced 
by the intrinsic goodness ti, but also fluctuate somewhat from one category to 
the next, due to the added e,j. If p = 1 then there is no fluctuation between 
categories and there is only one possible ranking of the universities for non- 
negative choice of weights c ^ 0. If p = then the cateogires are completely 
imcorrclated. Wc chose p = 0.6, because this gave higher correlation between 
category rankings on average, compared to the correlations observed in USNWR 
category rankings. 

Despite our conservative choice of p = 0.6, we still observed considerable 
variability in the possible overall rankings. Across 50 trials, the average ranking 
interval width for the 130 universities ranged between 20.02 and 28.19, and was 
22.90 on average. 

This shows that volatility in rankings is typical for seven dimensional data, 
even when the seven coordinates are influenced by a common variable. Intu- 
itively, this problem becomes worse as the dimension of the data increases. . 
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Figure 1: University rankings determined by choice of weights. The top three 
universities as determined by the choice of weights for three categories. The 
four cones comprise 99% of the total volume. 
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